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Background / Context: 

Success in middle school math is a key predictor of students’ success in high school and 
beyond — and is therefore an important issue for policy and practice in California and throughout 
the country. Middle school math coursetaking and success have clear consequences for the extent 
to which students reach advanced math courses — such as precalculus, calculus, trigonometry, or 
Advanced Placement math — before graduating from high school. Completing these advanced 
math courses can predict how well students are prepared for postsecondary-level math and 
whether they will be able to participate and succeed in regular college math courses without 
remediation or participation in developmental math courses. 

Mastering algebra is a critical step to enabling students to succeed in a college preparatory math 
sequence. But many students are unprepared to succeed in algebra, and they fail the course the 
first time they take it (Balfanz, McPartland, & Shaw, 2002; Finkelstein, Fong, Tiffany-Morales, 
Shields, & Huang, 2012; Huang, Snipes, & Finkelstein, 2014). The consequences of failing 
algebra can be considerable: only one in five students who fail algebra in grade 8 and repeat it in 
grade 9 achieves proficiency by the end of grade 9 (Finkelstein et al., 2012). Students not 
achieving algebra proficiency by the end of grade 9 have little chance of reaching and 
succeeding in advanced college preparatory math courses by the end of high school (Schiller & 
Muller, 2003; Spielhagen, 2006; Schiller, Schmidt, Muller, & Houang, 2010). 

Purpose / Objective / Research Question / Focus of Study: 

To raise math success rates in middle school, many schools and districts have implemented 
summer math programs designed to improve student preparation for algebra content in grade 8. 
However, little is known about the effectiveness of these programs. While students who 
participate typically experience learning gains, there is little rigorous evidence evaluating the 
effects of the programs on math achievement or readiness for algebra content. This study fills 
that void by rigorously examining the effects of one such summer program (the Elevate summer 
math program) on student achievement. In summer 2014 the Silicon Valley Education 
Foundation (SVEF), the research team, and several Silicon Valley school districts collaborated 
on a randomized controlled trial to assess the effects of the Elevate Math summer program on 
math achievement, algebra readiness, and attitudes toward math. The study focused on three 
primary questions: 

1 . What is the impact of the Elevate Math summer program on the math achievement and 
algebra readiness of rising grade 8 students? 

2. What is the impact of the Elevate Math summer program on math achievement in the 
math topic areas most closely aligned with the program’s curriculum? 

3. What is the impact of the Elevate Math summer program on the math interest and math 
self-efficacy of rising grade 8 students? 

Setting: 

The randomized controlled trial was conducted in summer 2014 at eight schools in six districts in 
California’s Silicon Valley. Participating districts identified eligible students based on existing 


SREE Spring 2016 Conference Abstract Template 


1 


grade 6 California Standards Test (CST) data. The districts’ enrollments range from 2,487 to 
13,162, with an average of 9,426. The percentage of English learner students in each district 
ranges from 19 percent to 53 percent, with an average of 38 percent. 

Population / Participants / Subjects: 

The average percentage of students in the participating districts eligible for free or reduced-price 
lunch is 57 percent. On average, 52 percent of the students are Hispanic, 34 percent are Asian, 9 
percent are White (non-Hispanic), and 2 percent are Black (non-Hispanic). Recruitment efforts 
by the Silicon Valley Education Foundation yielded 496 students to participate in the study. 
Thirty five students opted either out of data collection or lacked baseline data and were removed 
from the sample, leaving 461 students. Among these students, 56 percent ( n = 256) were male 
and 44 percent ( n = 205) were female. Though the Elevate Math summer program targeted 
students who scored at the high basic level or the low proficient level on the grade 6 math CST, 
all 496 grade 7 applicants from participating districts were accepted, regardless of prior math 
skills. More than half of participating students fell into the target range for the: 35 percent scored 
at the high basic level; 18 percent scored at the low proficient level; 29 percent scored below the 
target range for the intervention; and 17 percent scored above the range. 

Intervention / Program / Practice: 

Elevate Math is a math support program designed by the Silicon Valley Education Foundation as 
part of its ongoing effort to help students succeed in middle school math and to master important 
math and science skills that are needed to succeed in college and the labor market. Though the 
program is a year-round effort, its core is an intensive 75 -hour (19 days over four weeks) 
summer preparatory course. 

The Elevate Math summer program consists of four main components: 

• A Common Core State Standards-based curriculum that covers four math content 
modules: properties and operations, linear equations, ratios and multiple representations, 
and transfonnational geometry. 

• Approximately 19 days of four hours of blended learning classroom instruction, with one 
hour each day spent on Khan Academy (a free online learning system with thousands of 
educational resources). Each Khan Academy session includes a set of computer-based 
exercises that reflect the topics covered in the classroom that day. Students also have 
access to Khan Academy web-based videos to review any math topics covered during 
their class time. 

• A field trip to a local college or university, as well as a college information night for 
families and students to encourage college awareness. 

• Credentialed teachers and their college-level teacher assistants receive 40 hours of 
Common Core State Standards-based professional development provided by the Santa 
Clara County Office of Education and the Krause Center for Innovation. The first 24 
hours (which occur prior to the summer instruction) include training on curriculum 
understanding and implementation, instructional strategies aligned with the standards, 
math practices, technology integration in the classroom, and student engagement. The 
next 16 hours (which occur over the summer after instruction has begun) are spent in a 
professional learning community setting, where a coach facilitates the meeting to provide 
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a better understanding of specific Common Core State Standards instructional strategies 
and math practices that are useful to teach the Elevate Math curriculum. 

Research Design: 

Students were randomly assigned to a treatment group that received access to the program at the 
beginning of the summer or to a control group that received access to the program later in the 
summer. Random assignment was conducted separately for students in each program site. End- 
of-program test scores and survey responses of students in the treatment group were compared 
with those of students in the control group prior to their exposure to the program. Control group 
outcomes at the beginning of the second session are valid estimates of the outcomes that would 
have been observed among students in the treatment group, had they not had access to the 
program 

Data Collection and Analysis: 

The study examines the Elevate Math summer program’s effects on three sets of outcomes: math 
achievement and algebra readiness; achievement in specific math topic areas aligned with the 
Elevate Math summer program’s curriculum; and math interest and math self-efficacy. Math 
achievement was measured using the Mathematics Diagnostic Testing Project (MDTP) Algebra 
Readiness test, which was administered to the treatment and control groups on the first and last 
days of their participation in the summer program. The test consists of 45 multiple-choice items 
in seven topic areas. Previous research has shown that this test is highly predictive of success in 
grade 8 algebra I, and that grade 7 students have to pass three or more MDTP topic areas to have 
a greater than 50 percent chance of success in algebra I in grade 8 (Huang et ah, 2014). 

Therefore, for this study, math achievement was defined as the MDTP Algebra Readiness total 
score, and algebra readiness was defined according to whether students passed three or more 
MDTP topic areas. 

Math interest and math self-efficacy were assessed using measures drawn primarily from a 
student perception survey developed by Gilbert (2008) and administered by the Silicon Valley 
Education Foundation as part of its previous and ongoing program monitoring efforts. The math 
interest scale consisted of five items related to students’ interest in math (for example, how 
exciting math is to the students, how much the students like doing math). The math self-efficacy 
scale consisted of seven items related to students’ confidence in their ability to do math. 

Baseline equivalence was assessed by comparing performance on the grade 6 math CST between 
the treatment and control groups. Program effects were calculated by comparing treatment group 
outcomes at the end of the first summer session to control group outcomes at the beginning of 
the second summer session. Program effects were estimated using a single-level regression 
model comparing average outcomes in the treatment and control groups. The regression model 
controlled for grade 6 CST scores in order to improve the precision of the estimates and adjust 
for any differences in pre-program achievement. 

Findings / Results: 

The Elevate Math summer program significantly improved math achievement and algebra 
readiness among participating grade 7 students. The program improved the math achievement of 
the treatment group compared with the control group across several metrics. The treatment 
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group’s performance on the grade 7 MDTP Algebra Readiness test exceeded that of the control 
group. Out of 45 possible points, the average MDTP score for the treatment group was 21 points, 
compared with 17 points for the control group. This statistically significant difference of 4 points 
is the equivalent of 0.7 standard deviation. 

The Elevate Math summer program also had a positive, statistically significant effect on algebra 
readiness, as measured by whether students mastered at least three of seven MDTP topic areas. 
The program increased the percentage of students who passed at least three MDTP Algebra 
Readiness topic areas from 12 percent in the control group to 29 percent in the treatment group, 
an improvement of 142 percent compared to the control group. 

Despite the Elevate Math summer program’s effects, students’ math achievement at the end of 
the program suggested that many students were still not ready for the algebra content in grade 8 
math courses. At the end of the program, the average student in the treatment group correctly 
answered 21 out of 45 items (or 47 percent), and most students in the treatment group (nearly 70 
percent) had not reached achievement benchmarks associated with having a 50 percent or better 
chance of passing algebra I in grade 8. These findings suggest that, while the Elevate Math 
summer program had a positive effect on student achievement and readiness for algebra content, 
participating students will most likely need more support to succeed in grade 8 math courses 
with algebra content. 

There were no significant impacts on math interest or math self-efficacy. The estimated level of 
interest in math for the treatment group was higher than that of the control group, but the 
difference was not statistically significant; there was no evidence of positive effects on math self- 
efficacy 

Conclusions: 

This study affirms that the Elevate Math summer program improves math skills for students as 
they approach grade 8. The findings also show that the Elevate Math summer program works 
both by increasing students’ math skills and by helping students avoid summer learning loss 
through the first half of the summer. In fact, 37 percent of the program effect is due to the 
avoidance of summer learning loss among the treatment group. Unfortunately, the design of the 
study does not allow for examining whether the positive effects persist into the fall or beyond. 

By contrast, the findings suggest that the Elevate Math summer program has no effect on 
students’ sense of efficacy in math or their interest in the subject. 

Nevertheless, the average perfonnance of students who participated in the program indicates that 
most of them would not be prepared to succeed in algebra I. This suggests that summer programs 
such as Elevate Math may be important tools for improving math achievement among students 
entering grade 8, but that they are not sufficient by themselves for ensuring that students are 
ready for challenging middle school math courses. Most students like those who participated in 
this study, 82 percent of whom, scored in the low proficient category or below on the grade 6 
math CST, would need additional supports in order to succeed in grade 8 math courses and 
beyond. 
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